GH-50779: [C++][Parquet] Replace remaining RapidJSON usage with simdjson by Reranko05 · Pull Request #50781 · apache/arrow

Reranko05 · 2026-08-03T10:41:09Z

Rationale for this change

This PR continues the simdjson migration by replacing the remaining RapidJSON usages under cpp/src/parquet with simdjson and JsonWriter. It also removes the remaining unnecessary RapidJSON dependencies from the Parquet test target and Meson build configuration.

What changes are included in this PR?

Replace the remaining RapidJSON parsing logic in reader_test.cc with simdjson.
Replace RapidJSON parsing and serialization in geospatial/util_json_internal.cc with simdjson and JsonWriter.
Replace RapidJSON string escaping in types.cc with JsonWriter.
Remove the unused rapidjson_dep dependency from cpp/src/parquet/meson.build.
Remove the remaining RapidJSON dependency from the Parquet test target in CMakeLists.txt.
GitHub Issue: [C++][Parquet] Replace remaining RapidJSON usage with simdjson #50779

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-08-03T10:44:42Z

⚠️ GitHub issue #50779 has been automatically assigned in GitHub to PR creator.

Reranko05 · 2026-08-03T14:42:40Z

@rok @taepper

I'm migrating parquet/geospatial/util_json_internal.cc and ran into an ondemand question.

GeospatialGeoArrowCrsToParquetCrs() needs to:

inspect the crs object (e.g. check the id.authority/code fields to detect EPSG:4326/CRS84), and
if it isn't a recognized lon/lat CRS, serialize the same object back to JSON.

With RapidJSON this was straightforward because the DOM is reusable, but with simdjson::ondemand the object has already been consumed by the time I want to serialize it, leading to OUT_OF_ORDER_ITERATION.

Is the expected approach here to restructure the code into a single pass, or is there another way to serialize an already-inspected ondemand::object?

Thanks!

rok · 2026-08-03T15:08:43Z

As per docs here you could probably use the .reset() in case you don't get desired CRS fields.

  // Inspect crs_object...

  // If it was not a recognized lon/lat CRS:
  ARROW_ASSIGN_OR_RAISE(
      auto ignored,
      ::arrow::internal::GetSimdjsonResult(
          crs_object.reset(), "Failed to reset 'crs' object: "));

  ARROW_ASSIGN_OR_RAISE(
      auto raw_crs,
      ::arrow::internal::GetSimdjsonResult(
          crs_object.raw_json(), "Failed to get raw 'crs' JSON: "));

  return std::string(raw_crs);

Check also rewind and consider what's best for your usecase.

Reranko05 · 2026-08-03T16:38:13Z

Thanks @rok, reset() + raw_json() fixed the iteration issue.

However, one test still fails because raw_json() preserves the original formatting (e.g. {"key0": "value0"}), whereas the previous RapidJSON Writer minified it to {"key0":"value0"}. LogicalType::Equals() compares the CRS string literally.

Would you prefer keeping the previous compact formatting or preserving the original JSON formatting?

rok · 2026-08-03T16:55:05Z

@Reranko05 let's avoid behavior changes. Perhaps we can use simdjson::minify?

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Reranko05 · 2026-08-04T06:26:04Z

@kou @rok @pitrou Could you review this when you have time?

rok

Perhaps we can remove rapidjson from here also?

rok · 2026-08-04T14:43:02Z


 # Add RapidJSON & simdjson libraries
 list(APPEND PARQUET_SHARED_PRIVATE_LINK_LIBS RapidJSON simdjson::simdjson)
 list(APPEND PARQUET_STATIC_LINK_LIBS RapidJSON simdjson::simdjson)


Can we remove this yet?

rok · 2026-08-04T14:49:39Z

I also see many RapidJSON references in cpp/cmake_modules/ThirdpartyToolchain.cmake. Are we ready to remove those yet? (I don't know, hence the question)

kou · 2026-08-04T21:08:17Z

+  if (auto string = ::arrow::internal::GetJsonAs<std::string_view>(json_crs);
+      string.ok()) {
+    if (*string == "EPSG:4326" || *string == "OGC:CRS84") {


Can we avoid if (...; ...) for readability?

Suggested change

if (auto string = ::arrow::internal::GetJsonAs<std::string_view>(json_crs);

string.ok()) {

if (*string == "EPSG:4326" || *string == "OGC:CRS84") {

auto crs_string_result = ::arrow::internal::GetJsonAs<std::string_view>(json_crs);

if (crs_string_result.ok()) {

auto crs_string = *crs_string_result;

if (crs_string == "EPSG:4326" || crs_string == "OGC:CRS84") {

kou · 2026-08-04T21:15:29Z

+      auto code_string = ::arrow::internal::GetJsonAs<std::string_view>(code);
+
+      if (code_string.ok()) {
+        if ((authority_string == "OGC" && *code_string == "CRS84") ||
+            (authority_string == "EPSG" && *code_string == "4326")) {


How about using _result suffix for arrow::Result variable for readability?

Suggested change

auto code_string = ::arrow::internal::GetJsonAs<std::string_view>(code);

if (code_string.ok()) {

if ((authority_string == "OGC" && *code_string == "CRS84") ||

(authority_string == "EPSG" && *code_string == "4326")) {

auto code_string_result = ::arrow::internal::GetJsonAs<std::string_view>(code);

if (code_string_result.ok()) {

auto code_string = *code_string_result;

if ((authority_string == "OGC" && code_string == "CRS84") ||

(authority_string == "EPSG" && code_string == "4326")) {

kou · 2026-08-04T21:21:32Z

-    json_crs.Accept(writer);
-    return buffer.GetString();
+  RETURN_NOT_OK(::arrow::internal::GetSimdjsonResult(crs_object.reset(),
+                                                     "Failed to reset 'crs' object: ")


It seems that all GetSimdjsonResult() messages have : suffix. How about adding it automatically?

diff --git a/cpp/src/arrow/util/simdjson_internal.h b/cpp/src/arrow/util/simdjson_internal.h index 8ffb741da4..187dd3aaa5 100644 --- a/cpp/src/arrow/util/simdjson_internal.h +++ b/cpp/src/arrow/util/simdjson_internal.h @@ -85,7 +85,7 @@ template <typename T> Result<T> GetSimdjsonResult(simdjson::simdjson_result<T> result, std::string_view error) { T value; if (auto error_code = std::move(result).get(value); error_code != simdjson::SUCCESS) { - return Status::Invalid(error, simdjson::error_message(error_code)); + return Status::Invalid(error, ": ", simdjson::error_message(error_code)); } return value; }

(BTW, GetSimdjsonResult() name may be a bit strange because it doesn't return simdjson::result<T>. It gets arrow::Result<T> from simdjson::result<T>. ResolveSimdjsonResult(), SimdjsonResultToArrow() or something may be better.)

kou · 2026-08-04T21:29:15Z

+    ::arrow::json::JsonWriter writer;
+    writer.String(crs);
+
+    auto escaped = writer.GetString().ValueUnsafe();


Should we change the return type to arrow::Result<std::string> from std::string to propagate an error?

kou · 2026-08-04T21:33:04Z

-  json << R"(, "crs": )" << buffer.GetString();
+  ::arrow::json::JsonWriter writer;
+  writer.String(crs);
+  json << R"(, "crs": )" << writer.GetString().ValueUnsafe();


Can we use JsonWriter for all JSON build instead of mixing manual JSON build and JsonWriter build?

Copilot AI lite review requested due to automatic review settings August 3, 2026 10:41

Reranko05 requested review from pitrou and wgtmac as code owners August 3, 2026 10:41

Copilot AI reviewed Aug 3, 2026

github-actions Bot added the awaiting review Awaiting review label Aug 3, 2026

github-actions Bot added Component: Parquet Component: C++ labels Aug 3, 2026

Reranko05 marked this pull request as draft August 3, 2026 10:55

Reranko05 added the CI: Extra: C++ Run extra C++ CI label Aug 3, 2026

Replace remaining RapidJSON usage with simdjson

0230946

Reranko05 force-pushed the gh-35460-pr5-updated branch from d026d1f to 0230946 Compare August 3, 2026 12:54

Preserve GeoArrow CRS JSON formatting

777999e

Reranko05 force-pushed the gh-35460-pr5-updated branch from 270af67 to 777999e Compare August 4, 2026 05:40

Reranko05 marked this pull request as ready for review August 4, 2026 06:23

Copilot AI review requested due to automatic review settings August 4, 2026 06:23

Copilot AI reviewed Aug 4, 2026

rok reviewed Aug 4, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 4, 2026

kou reviewed Aug 4, 2026

View reviewed changes

-  if (auto string = ::arrow::internal::GetJsonAs<std::string_view>(json_crs);
-      string.ok()) {
-    if (*string == "EPSG:4326" || *string == "OGC:CRS84") {
+  auto crs_string_result = ::arrow::internal::GetJsonAs<std::string_view>(json_crs);
+  if (crs_string_result.ok()) {
+    auto crs_string = *crs_string_result;
+    if (crs_string == "EPSG:4326" || crs_string == "OGC:CRS84") {

-      auto code_string = ::arrow::internal::GetJsonAs<std::string_view>(code);
-      if (code_string.ok()) {
-        if ((authority_string == "OGC" && *code_string == "CRS84") ||
-            (authority_string == "EPSG" && *code_string == "4326")) {
+      auto code_string_result = ::arrow::internal::GetJsonAs<std::string_view>(code);
+      if (code_string_result.ok()) {
+        auto code_string = *code_string_result;
+        if ((authority_string == "OGC" && code_string == "CRS84") ||
+            (authority_string == "EPSG" && code_string == "4326")) {

Uh oh!

Conversation

Reranko05 commented Aug 3, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

Reranko05 commented Aug 3, 2026

Uh oh!

rok commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reranko05 commented Aug 3, 2026

Uh oh!

rok commented Aug 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reranko05 commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rok left a comment

Choose a reason for hiding this comment

Uh oh!

rok Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

rok commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

kou Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Reranko05 commented Aug 3, 2026 •

edited by github-actions Bot

Loading

rok commented Aug 3, 2026 •

edited

Loading

Reranko05 commented Aug 4, 2026 •

edited

Loading

rok commented Aug 4, 2026 •

edited

Loading